A new IRT-based standard setting method: application to eCat-listening.
نویسندگان
چکیده
BACKGROUND Criterion-referenced interpretations of tests are highly necessary, which usually involves the difficult task of establishing cut scores. Contrasting with other Item Response Theory (IRT)-based standard setting methods, a non-judgmental approach is proposed in this study, in which Item Characteristic Curve (ICC) transformations lead to the final cut scores. METHOD eCat-Listening, a computerized adaptive test for the evaluation of English Listening, was administered to 1,576 participants, and the proposed standard setting method was applied to classify them into the performance standards of the Common European Framework of Reference for Languages (CEFR). RESULTS The results showed a classification closely related to relevant external measures of the English language domain, according to the CEFR. CONCLUSIONS It is concluded that the proposed method is a practical and valid standard setting alternative for IRT-based tests interpretations.
منابع مشابه
Group Decision Making based on a New Evaluation Method and Hesitant Fuzzy Setting with an Application to an Energy Planning Problem
In recent two decades, countries focused on minimum extraction of fossil fuels and utilized the renewable energies based on countries' policies and the environmental considerations. Thus, choosing the best renewable energy alternative is a significant role to investment on them. Among the classical decision approaches that have used in the literature, the hesitant fuzzy sets (HFSs) theory is ap...
متن کاملeCAT-Listening: design and psychometric properties of a computerized adaptive test on English Listening.
In this study, eCAT-Listening, a new computerized adaptive test for the evaluation of English Listening, is described. Item bank development, anchor design for data collection, and the study of the psychometric properties of the item bank and the adaptive test are described. The calibration sample comprised 1.576 participants. Good psychometric guarantees: the bank is unidimensional, the items ...
متن کاملNursing students experiences toward evidence-based learning in clinical setting: A qualitative study
Introduction: Evidence-based learning is an effective educational strategy. Assessment of nursing student’s viewpoints about this new method can increase its application. This research was conducted to find out student’s experiences toward evidence-based learning in clinical setting. Methods: This qualitative study was performed by grounded theory method in 2010. Eighteen nursing students were...
متن کاملDevelopment, psychometric properties and new validity evidences of the web-based computerized adaptive test of English eCat
This paper describes the process of constructing the eCat test and provides information on the effects its normal operation had on its psychometric properties. Results obtained from 3224 students revealed that the 225 item pool has adequate reliability and good convergent validity with respect to self-reported English proficiency. Simulations showed that ability estimations are essentially unbi...
متن کاملA New Balancing and Ranking Method based on Hesitant Fuzzy Sets for Solving Decision-making Problems under Uncertainty
The purpose of this paper is to extend a new balancing and ranking method to handle uncertainty for a multiple attribute analysis under a hesitant fuzzy environment. The presented hesitant fuzzy balancing and ranking (HF-BR) method does not require attributes’ weights through the process of multiple attribute decision making (MADM) under hesitant conditions. For the rating of possible alternati...
متن کاملذخیره در منابع من
با ذخیره ی این منبع در منابع من، دسترسی به آن را برای استفاده های بعدی آسان تر کنید
برای دانلود متن کامل این مقاله و بیش از 32 میلیون مقاله دیگر ابتدا ثبت نام کنید
ثبت ناماگر عضو سایت هستید لطفا وارد حساب کاربری خود شوید
ورودعنوان ژورنال:
- Psicothema
دوره 25 2 شماره
صفحات -
تاریخ انتشار 2013